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Generally  r-jan  instruction  was  read  faster  if  its  topics  were  mentioned  in 
the  order  in  which  they  would  be  executed.  This  effect,  taken  together 
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Experiment  2  used  the  same  material.':  as  the  first  study,  but  now  the 
subject's  task  was  simply  to  memorize  each  instruction.  The  time  spent 
reading  each  word  was  measured.  There  was  only  limited  evidence  for  use 
of  a  step  schema,  likely  because  the  memory  requirement  fostered  a  task- 
specific  strategy. 


In  Experiment  3,  we  switched  to  an  assembly  task.  Subjects  had  to 
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of  topics  in  a  step  (which  we  assumed  corresponded  to  the  number  of  slots 
in  a  step  schema  that  need  to  be  filled) .  Instructions  were  read  faster 


when  they  contained  fewer  topics  (or  schema  slots) ,  and  when  their  topics 
were  mentioned  in  the  order  they  would  be  executed.  Moreover,  these  effect 
were  more  pronounced  for  readers  who  had  smaller  working-memory  spans. 

These  results  were  consistent  with  the  idea  that  instruction  understanding 
involves  schema  instantiation. 


Tn  Experiment  4,  we  returned  to  a  device-operation  task,  but  used  a 
more  complex  device  than  that  used  in  the  earlier  studies.  Now,  we 
focused  on  information  other  than  that  in  the  instructions,  as  the  major 
variable  was  whether  or  not  subjects  had  learned  a  mental  model  of  the 
device  they  had  to  operate.  Having  a  mental  model  facilitated  memory  of 
the  components  of  the  device,  as  well  as  the  time  to  read  an  instruction 
about  operating  the  device,  as  well  as  ability  to  reason  about  the  device 
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Abstract 

This  project  dealt  with  how  people  understand  and  use 
written  instructions  in  operating  a  device  or  assembling  an 
object.  In  Experiment  1,  subjects  read  and  then  executed 
single-step  instructions  for  operating  a  simple  device  (e.g., 
"Turn  the  right  knob  so  that  the  gamma  meter  reads  20  if  the 
sigma  indicator  light  is  on").  Each  instruction  contained  three 
topics  —  an  antecedent  (e.g.  ,  "sigma  indicator  on"),  an  action 
(e.g.,  "turn  the  right  knob"),  and  a  consequence  (e.g.,  "gamma 
meter  reads  20")  —  and  the  order  in  which  the  topics  were 
mentioned  varied  across  instructions.  Generally,  an  instruction 
was  read  faster  if  its  topics  were  mentioned  in  the  order  in 
which  they  would  be  executed.  This  effect,  taken  together  with 
results  due  to  individual  differences  in  working  memory, 
suggested  that  understanding  an  instruction  involves 
instantiating  a  step  schema ,  i.e.,  an  abstract  representation  of 
an  instructional  step  that  contains  slots  for  antecedent,  action, 
and  consequence  information. 

Experiment  2  used  the  same  materials  as  the  first  study,  but 
now  the  subject’s  task  was  simply  to  memorize  each  instruction. 
The  time  spent  reading  each  word  was  measured.  There  was  only 
limited  evidence  for  use  of  a  step  schema,  likely  because  the 
memory  requirement  fostered  a  task-specific  strategy. 

In  Experiment  3,  we  switched  to  an  assembly  task.  Subjects 
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had  to  assemble  a  model  helicopter  from  instructions  that 
included  72  steps.  Each  step  was  presented  one  at  a  time  (e.g., 
"Slide  another  joint  onto  the  block  so  that  the  narrow  end  faces 
upward").  Again  we  varied  the  order  in  which  the  various  topics 
in  a  step  were  mentioned  (where  now  topics  include  apent .  action, 
object .  location.  etc.);  in  addition,  we  varied  the  number  of 
topics  in  a  step  (which  we  assumed  corresponded  to  the  number  of 
slots  in  a  step  schema  that  need  to  be  filled).  Instructions 
were  read  faster  when  they  contained  fewer  topics  (or  schema 
slots).  and  when  their  topics  were  mentioned  in  the  order  they 
would  be  executed.  Moreover,  these  effects  were  more  pronounced 
for  readers  who  had  smaller  working-memory  spans.  These  results 
were  consistent  with  the  idea  that  instruction  understanding 
involves  schema  instantiation. 

In  Experiment  4,  we  returned  to  a  device-operation  task,  but 
used  a  more  complex  device  than  that  used  in  the  earlier  studies. 
Now,  we  focused  on  information  other  than  that  in  the 
instructions,  as  the  major  variable  was  whether  or  not  subjects 
had  learned  a  ment  a  I  mode  1  of  the  device  they  had  to  operate. 
Having  a  mental  model  facilitated  memory  of  the  components  of  the 
device,  as  well  as  the  time  to  read  an  instruction  about 
operating  the  device,  as  well  as  ability  to  reason  about  the 
devi ce  . 
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1  .  BACKGROUND 

1.1  Major  Goal  s 

The  overall  goal  of  our  project  was  to  understand  the 
cognitive  processes  operative  when  people  use  written 
instructions  as  an  aid  to  assembling  an  object  or  operating  a 
device.  By  understanding  such  processes,  we  hoped  to  contribute 
to  a  general  theory  of  instruction  as  well  as  to  understand  the 
sources  of  individual  differences  in  peoples’  abilities  to 
comprehend  instructions. 

In  addition  to  the  above  basic-research  goals,  there  were 
also  practical  goals  of  our  work.  One  was  to  provide  a 
scientific  basis  for  making  guidelines  for  the  construction  of 
real-world  instructions,  such  as  training  manuals  and  job  aides. 
Another  practical  goal  concerned  individual  differences:  by 
relating  differences  in  how  well  different  people  coped  with 
written  instructions  to  measurable  differences  in  their 
information  processing  capacities,  we  hoped  to  make  it  possible 
to  screen  individuals  more  effectively  for  different  kinds  of 
training  materials . 

1.2  Two  Foci  of  Instructions  Research 

Since  the  inception  of  our  work  on  instructions,  there  have 
been  two  distinct  foci  of  our  research.  One  focus  is  on  the 
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instructions  themselves;  the  second  is  on  knowledge  other  than 
the  instructions  that  individuals  bring  to  bear  in  assembling  an 
object  or  operating  a  device.  We  can  make  this  contrast  clearer 
by  considering  a  case  where  one  is  assembling  a  model  helicopter: 
one  may  try  to  do  this  solely  by  following  the  instructions  that 
accompany  the  kit;  alternatively,  one  may  try  to  assemble  the 
model  without  ever  consulting  the  accompanying  instructions, 
relying  instead  on  what  the  model  is  supposed  to  look  like  when 
completed,  and  on  one's  past  experience  in  assembling  models. 
Another  way  to  highlight  the  difference  between  our  two  research 
foci  is  to  note  the  different  general  research  areas  that  they 
are  related  to;  when  we  focus  on  the  instructions  per  se .  our 
work  seems  most  related  to  text  processing;  when  we  focus  on 
other  knowledge  about  the  task,  our  work  seems  more  related  to 
procedural  learning  and  problem  solving. 

Over  the  course  of  our  work  on  instructions,  we  have  shifted 
from  one  focus  to  another.  Our  earlier  work  (summarized  in  Smith 
&  Goodman,  1984)  dealt  with  the  assembly  of  electrical  circuits. 
Our  major  concern  then  was  how  explanatory  material  about 
circuits  in  general  affected  one's  ability  to  understand  and  use 
instructions  to  assemble  a  specific  circuit;  this  was  a  clear-cut 
case  of  focusing  on  knowledge  other  than  the  instructions 
themselves.  We  found  that  providing  subjects  with  an  explanation 
of  either  the  structure  or  function  of  circuits  decreased  the 
time  needed  to  read  each  step.  These  results  are  illustrated  in 
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Figure  1.  Note  that  while  explanatory  materiel  partly  determines 
the  time  to  read  a  step,  there  are  other  determinants  of  reading 
time  that  are  intrinsic  to  the  step;  i.e.,  the  reading  time 
curves  rise  and  fall  for  different  steps  in  the  same  way  for  all 
groups.  This  led  us  to  look  at  the  intrinsic  determinants  of 
step  difficulty,  which  is  an  instance  of  focusing  on  the 
instructions  themselves.  We  found  that  the  time  to  read  a  step 
increased  with  the  number  of  propositions  it  contained.  Further 
analysis  suggested  that  some  propositions  were  more  important 
than  others;  two  propositions  that  dealt  with  different  topics 
(e.g.,  wires,  dry  cells)  contributed  more  to  reading  time  than 
two  that  described  a  single  topic  (eg.,  the  color  and  size  of  a 
wire).  Thus,  something  like  the  notion  of  topics  seemed  to  be  a 
proper  unit  for  studying  step  difficulty. 

In  the  first  phase  of  the  present  project  (e.g.,  Spoehr , 
Smith,  &  Morris,  1983;  1984),  we  shifted  our  focus  entirely  to 
the  instructions  themselves,  and  to  the  determinants  of  step 
difficulty.  Now  we  used  an  operation  task  (originated  by  Dixon, 
1982),  in  which  subjects  read  and  executed  instructions  about  how 
to  operate  a  simple  device.  Each  instruction  mentioned  three 
topics  (an  antecedent,  action,  and  consequence)  and  we  were 
interested  in  the  effects  on  reading  time  of  presenting  these 
topics  in  various  orders.  We  found  that  reading  time  was  fastest 
when  the  topics  were  mentioned  in  the  order  in  which  they  would 
be  executed  (antecedent,  action,  consequence). 
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In  the  last  year  of  the  present  project,  both  foci  of 
research  have  been  well  represented.  We  have  studied  two  tasks, 
assembly  of  a  model  helicopter  and  operation  of  a  control  panel. 
The  assembly  task  focused  almost  exclusively  on  the  instructions 
per  se .  while  the  operation  task  was  primarily  concerned  with  the 
effects  of  knowledge  other  than  the  instructions.  These  tasks 
again  showed  that  the  time  to  read  and  understand  instructions  is 
determined  both  by  factors  intrinsic  to  the  steps  themselves, 
such  as  the  number  and  order  of  topics,  as  well  as  by  explanatory 
material.  Virtually  all  of  the  remainder  of  this  report  is 
devoted  to  a  detailed  review  of  the  research  conducted  in  the 
past  two  years  . 

1.3  A  Comment  on  Individual  Differences 

Before  beginning  our  review,  a  word  is  in  order  about  the 
role  that  individual  differences  plays  in  our  experiments.  In 
all  our  studies,  we  obtained  measures  of  subjects’  memory 
abilities;  we  did  this  because  the  relation  between  (i) 
individual  differences  in  memory  and  (ii)  performance  on  an 
instructions  task  provides  important  clues  about  the  basic 
processes  operative  in  the  instructions  task.  The  individual- 
difference  data  also  have  their  applied  value,  in  that  they  can 
be  used  to  assess  what  kinds  of  abilities  are  needed  for  success 
in  different  kinds  of  tasks. 
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2.  SCHEMAS  AND  A  SIMPLE  OPERATIONS  TASK 

2 . 1  Rat i onal e 

Our  first  year  of  work  on  the  current  project  focused  on  the 
comprehension  of  simple  written  instructions  like  "If  the  sigma 
indicator  light  is  on,  turn  the  right  knob  in  order  to  make  the 
gamma  meter  read  20".  We  hypothesized  that  comprehension  of  such 
an  ins t rue i t ona 1  step  could  be  viewed  as  instantiating  a 
step-schema,  where  the  step-schema  of  interest  has  three  major 
slots  corresponding  to  the  action.  the  antecedent  (the 
circumstances  under  which  the  action  is  to  be  performed),  and  the 
consequence  (the  end  result  of  the  action).  Understanding  the 
preceding  sentence,  then,  amounts  to  filling  the  step-schema’s 
action  slot  with  "turn  right  knob",  its  antecedent  slot  with 
"sigma  indicator  light  on",  and  its  consequence  slot  with  "gamma 
meter  read  20".  The  question  of  major  interest  was,  How  should 
the  information  (topics)  in  an  instruction  be  ordered  so  as  to 
facilitate  the  process  of  instantiating  a  step-schema? 

2.2  Experiment  1:  Operating  a  Simple  Device 

2.2.1  Method 

Subjects  were  asked  to  read  and  execute  instructions  in 
order  to  operate  the  controls  of  the  simple  panel  shown  in  Figure 
2.  All  instructions  contained  three  slot-fillers,  or  components 
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of  information:  antecedent,  action,  and  consequence.  Each 


instruct i on 

was  one 

of 

three 

possible  orders  of 

the 

three 

components , 

chosen 

so 

that 

we  could  experimentally 

test 

three 

di f  f  er ent 

accounts 

of 

what 

ordering  would  yield 

the 

best 

performance.  According  to  the  first  account,  information  within 
a  step  schema  is  organized  with  the  action  being  primary  and  the 
other  slot  information  being  secondary  to  it  (Dixon,  19B2); 
therefore  an  action-consequence-antecedent  order  (e.g.,  "Turn  the 
right  knob  in  order  to  make  the  gamma  meter  read  20  if  the  sigma 
indicator  light  is  on.")  should  be  the  easiest  to  read. 
According  to  the  second  account,  the  slots  in  a  step  schema 
should  be  filled  in  the  order  in  which  they  are  needed  when  the 
step  is  executed;  therefore,  the  antecedent-action-consequence 
order  (e.g.,  "If  the  sigma  indicator  light  is  on  turn  the  right 
knob  in  order  to  make  the  gamma  meter  read  20.")  should  yield  the 
best  performance.  Finally,  the  third  account  has  it  that  schema 
slots  are  organized  by  goals,  as  it  is  in  many  production-system 
based  expert  systems;  hence  the  consequence— act i on-antecedent 
order  (e.g.,  "In  order  to  make  the  gamma  meter  read  20  turn  the 
right  knob  if  the  sigma  indicator  light  is  on")  should  be  best 

We  were  not  only  interested  in  finding  out  how  information 
in  step  schemas  is  structured  but  also  in  examining  how  the 
comprehension  process  is  affected  by  different  methods  of 
presentation.  Our  subjects  therefore  read  and  executed  a  series 
of  these  individual  steps  under  both  simultaneous  conditions, 
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where  the  instruction  and  device  display  appeared  at  the  sane 
time,  and  under  sequential  conditions,  where  the  instruction  was 
read  and  disappeared  before  the  device  display  appeared. 

2.2.2  Results 

2.2.2. 1  Basic  results 

Although  the  order  in  which  the  components  appeared  had  no 
effect  on  execution  times  under  either  presentation  method,  the 
component  order  did  affect  the  reading  times.  Under  simultaneous 
display  conditions,  the  antecedent-action-  consequence  order  was 
read  most  quickly,  followed  by  the  action-  consequence-antecedent 
order,  and  finally  by  the  consequence-action-  antecedent  order. 
This  suggested  that  components  were  being  filled  into  the  slots 
of  the  step  schema  in  the  same  order  they  were  needed  during  the 
execution  of  the  step.  In  the  sequential  condition,  less 
substantial  effects  of  slot-filler  ordering  were  found.  The 
consequence-action-  antecedent  form  still  took  the  lohgest  to 
read,  but  there  were  no  significant  differences  between  the  other 
two  orders.  Subjects  therefore  appeared  to  be  processing  the 
instructions  in  an  entirely  different  manner  when  they  had  to 
read  in  order  to  execute  a  step  from  memory. 

2.22.2  Individual  differences 

The  reasons  for  the  different  pattern  of  results  under  the 
two  presentation  conditions  become  apparent  when  one  considers 
the  working-memory  capacity  of  the  subjects.  We  consider 
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working-memory  a  critical  factor  in  a  reader’s  ability  to 
comprehend  instructions  because  of  its  involvement  in  just  about 
every  aspect  of  the  task.  At  the  simplest  level,  working-memory 
is  used  as  a  temporary  store  for  individual  words  and  phrases 
while  the  relevant  schema  is  being  operated  on.  Working-memory 
must  also  serve  as  a  store  for  the  reader’s  currently  active  set 
of  goals  and  plans  during  both  reading  and  execution,  and  may 
also  maintain  a  subset  of  the  reader’s  permanent  knowledge  store 
that  is  immediately  relevant  to  the  task. 

We  measured  working-memory  capacity  in  two  ways.  First  we 
administered  e  relatively  standard  memory-span  test:  we 
determined  the  maximum  list  length  of  digits  that  the  subject 
could  reliably  recall  in  reverse  order  immediately  after  hearing 
the  list.  We  will  refer  to  this  measure  as  the  "backward  digit" 
span.  Unfortunately,  the  psychological  literature  does  not  lend 
much  support  to  the  notion  that  a  standard  memory-span  measure 
such  as  backward-digit  span  is  at  all  predictive  of  reading 
comprehension  performance.  (see,  e.g.,  Crowder,  1982).  We 
theiefore  also  used  another  measure,  devised  by  Daneman  and 
Carpenter  (1980),  and  shown  to  correlate  well  with  performance  on 
reading  tasks.  Daneman  and  Carpenter  required  subjects  to  read 
aloud  sentences  of  14-16  words,  and  to  remember  the  final  word  in 
each  of  a  series  of  such  sentences.  The  working  memory  span  is 
the  longest  number  of  sentences  that  the  subject  can  read  in 
succession  and  still  reliably  recall  the  final  words  in  the 
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correct  ordpr . 

We 

will  refer  to  this 

capacity 

measure  as  the 

"working  memory' 

1  span. 

The  subjects 

in 

our 

experiment 

had 

working -memory 

spans 

ranging  from 

2 

t  0 

6 ,  and 

the  i  r 

work i ng-memory 

spans 

correlated  only 

slightly 

wi  th 

the  i  r 

backward-digit  spans. 

When  the  reading  times  for  steps  with  different  component 
orderings  are  considered  separately  for  subjects  with  different 
working-memory  spans,  an  interesting  pattern  emerges.  Under  the 
simultaneous  presentation  condition,  all  types  of  subjects  read 
the  antecedent-action-consequence  version  fastest,  the  action- 
consequence-antecedent  version  next  fastest,  and  the  consequence- 
action-antecedent  version  slowest.  The  higher  the  working-memory 
span,  the  faster  the  reading  times  overall,  but  there  was  no 
interaction  between  working-memory  span  and  ordering  of 
components.  The  pattern  was  quite  different  in  the  sequential 
condition.  Here,  although  the  subject's  with  the  higher 

working-memory  spans  were  faster  overall,  there  was  an 
interaction  between  component  order  and  working-memory  span. 
Subjects  with  good  capacity  read  all  types  of  instructions 
approximately  equally  fast,  but  subjects  with  poor  working-memory 
spans  showed  the  same  pattern  of  increasing  reading  times  over 
instruction  types  that  they  displayed  in  the  simultaneous 
cond i t i on . 

We  interpret  these  results  to  mean  that  the  simultaneous 
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presentation  of  instruction  and  display  allows  the  reader  to  scan 
the  instruction  for  the  component  that  is  to  be  executed  first 
(in  this  case,  the  antecedent),  and  to  actually  carry  it  out 
(evaluate  the  antecedent).  The  reader  can  then  scan  and  locate 
the  second  component  (the  action),  and  " ant i c i pa t or i 1 y  execute" 
it  in  the  sense  that  he  or  she  can  place  one  hand  on  the  proper 
control.  Finally  the  last  (consequence)  component  is  located  and 
read  prior  to  the  subject  signalling  the  completion  of  reading 
and  starting  to  actually  manipulate  the  controls.  The  net  result 
is  that  simultaneous  presentation  places  very  little  burden  on  a 
subject’s  working  memory,  making  it  possible  for  even  the  poorer 
capacity  subjects  to  use  the  same  strategy  that  the  better 
sub j  ec  t  s  use  . 

When  a  step  must  be  read  and  remembered  in  the  sequential 
condition,  subjects  must  instantiate  a  schema  while  they  are 
reading  each  instruction,  and  then  maintain  then  instantiated 
schema  in  memory  when  the  instruction  has  been  removed.  The 
poorer  working-memory  subjects  appear  to  be  processing  component 
by  component,  scanning  for  the  slot-fillers  in  the  order  in  which 
they  are  to  be  executed  in  much  the  same  way  as  they  did  under 
simultaneous  conditions  (though  in  this  case  there  can  be  no 
anticipatory  executions).  The  subjects  with  better  working- 
memories  appear  to  be  able  to  take  the  components  in  any  order  in 
which  they  are  presented  in  the  written  instruction  and  fit  them 
into  the  step  schema.  Thus  the  two  types  of  subjects  are  doing 

qualitatively  different  things  in  the  sequential  condition. 
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2.2.3  Summary 

To  summarize  our  findings,  work i ng -memory  limitations  appear 
to  force  subjects  into  using  different  reading  processes  when  the 
nature  of  the  task  creates  an  overload  on  working-memory.  The 
best  presentation  method  is  to  have  the  instruction  and  the 
display  coincide  in  time.  The  best  way  to  present  the  components 
within  a  single  step  is  to  put  them  in  the  order  in  which  they 
will  be  needed  when  the  step  is  executed.  Subjects  with  good 
working-memory  capacity  are  affected  little  by  deviations  from 
these  principles,  but  subjects  with  poor  capacity  are  impeded  to 
a  progressively  greater  extent  the  more  conditions  deviate  from 
the  opt imal . 

2.3  Experiment  2:  On-Line  Measures  of  Device  Operation 

2.3.1  Rati onal e 

In  this  experiment  we  used  the  same  materials  as  the 
preceding  study,  but  now  we  measured  the  reading  time  for  each 
word  in  an  instructional  step  as  our  goal  was  to  obtain  an 
on-line  analysis  of  how  a  step  is  understood.  Our  expectations 
about  the  patterns  of  reading  times  were  guided  by  our  schema 
account  of  understanding  instructions.  Specifically,  we  assumed 
that  understanding  a  simple  operation  instruction  amounts  to 
instantiating  a  schema  with  slots  for  the  antecedent,  action,  and 
consequence,  and  that  these  slots  need  to  be  filled  in  the  order 
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just  specified.  Given  this  view,  and  the  assumption  that  filling 
a  schema  slot  takes  measurable  t;me,  it  follows  that: 

(1)  When  the  order  of  components  is  antecedent-act i on- 
consequence  ,  reading  times  should  increase  at  words  that  end 
components,  because  the  reader  should  be  filling  the  relevant 
schema  slots  at  these  points. 

(2)  With  other  orders  of  components,  readers  may  not  pause 
at  the  end  of  a  slot-filler  if  it  does  not  correspond  to  the 
schema  slot  that  the  reader  is  searching  for;  in  these  cases, 
though,  the  reader  may  spend  a  relatively  long  duration  on  each 
word  because  he  is  holding  a  relatively  large  amount  of 
information  in  working  memory. 

In  addition  to  the  above,  we  explored  a  third  hypothesis,  which 
was  not  tied  to  our  schema  account: 


(3)  For 

any 

order 

o  f 

components,  reading 

times 

should 

be 

relatively 

1  ong 

for  . 

the 

"informat ive” 

words , 

i  .  e  .  , 

words 

that 

change  from 

i ns  true  t i on 

to 

i ns  t  rue  t i on ; 

there 

were 

f  ive 

such 

words  in  each  instruction,  including  the  name  of  the  relevant 
indicator  light  (e.g.,  "theta",  "sigma"),  whether  it  was  "on"  or 
"off",  whether  the  knob  to  be  turned  was  "left"  or  "right",  the 
name  of  the  relevant  meter  (e.g.,  "alpha",  "beta"),  and  that 
meter's  reading. 
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2.3.2  Method 

Since  this  study  was  an  exploratory  one,  we  decided  to 
expedite  matters  by  using  a  simple  recall  task  rather  than  our 
device-operation  task.  Subjects  read  for  understanding  each 
instructional  step,  and  then  tried  to  recall  it  verbatim.  In 
previous  research  (Spoehr  et  al  .  ,  1983),  we  found  that  this 
recall  task  reveals  the  same  effect  of  order  of  components  as 
does  the  device-operation  task,  presumably  because  the  same 
schema  is  employed  in  both  tasks. 

Thirty  different  instructions  were  used.  In  10  of  them,  the 
order  of  components  was  antecedent-action-consequence  (the 
"antecedent-first"  condition);  in  another  10,  the  order  was 
action-consequence-antecedent  ("action-first");  and  in  the 
remaining  10,  the  order  was  consequence-action-antecedent 
("consequence-first").  The  three  types  of  instructions  were 
randomly  intermixed.  On  each  trial,  an  instruction  was  presented 
on  a  microprocessor  display  one  word  at  a  time,  and  subjects 
pressed  a  space  bar  as  soon  as  they  understood  the  current  word 
and  its  relation  to  the  previous  ones.  Depression  of  the  space 
bar  immediately  triggered  presentation  of  the  next  word.  After 
the  last  word  of  the  sentence  went  off,  subjects  attempted  to 
recall  the  sentence  in  correct  order. 
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2.3.3  Results 

While  our  primary  interest  is  in  the  reading  times,  we  also 
analyzed  the  recall  data.  Only  the  five  informative  words  in 
each  sentence  were  scored  for  accuracy.  The  percentage  of  words 
recalled  in  correct  order  was  95.2  for  the  antecedent-first 
condition,  91.4  for  the  consequence-first  condition,  and  85.0  for 
the  action-first  condition.  These  results  reveal  another 
performance  advantage  of  the  antecedent-action-consequence  order. 

Let  us  move  on  to  the  reading  times.  As  in  our  previous 
recall  study  (Spoehr  et  al . ,  19B3),  instructions  in  the 
antecedent-first  order  were  read  faster  than  those  in  the 
action-first  order.  In  the  present  study,  however,  there  was  no 
real  difference  between  the  antecedent-first  and  the  consequence- 
first  orders.  More  detailed  analyses  are  in  Figure  3,  which 
presents  the  reading  times  (measured  in  milliseconds)  for  each 
word  of  an  instruction,  separately  for  the  three  orders.  (Note 
that  the  words  in  the  sample  instruction  given  at  the  top  of  each 
panel  are  not  always  perfectly  alligned  with  their  position  in 
the  sentence  as  given  on  the  x-axis.)  Consider  how  these  results 
line  up  with  our  hypotheses  about  increased  processing  at  the  end 
of  components  or  slot  fillers.  For  the  antecedent-first 
condition  (top  panel  of  Figure  3),  reading  times  are  relatively 
long  at  the  words  ending  the  antecedent  and  consequence 
components,  but  not  for  the  word  ending  the  action  component. 
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For  the  action-first  and  consequence-first  orders  (the  middle  and 
bottom  panels  of  Figure  3),  reading  times  are  elevated  only  for 
words  ending  the  antecedent  component.  Though  this  latter 
outcome  is  consistent  wit.h  our  second  hypothesis  (the  reader  is 
presumably  searching  for  the  antecedent  slot-filler),  there  is 
reason  for  caution.  Specifically,  in  the  action-first  and 
consequence-first  orders,  the  last  word  of  the  antecedent  is  also 
the  last  word  of  the  sentence,  so  the  pause  at  the  end  of  these 
antecedents  may  reflect  some  end-of-sentence  process  that  occurs 
in  all  condi t i ons . 

Our  third  hypothesis  was  that  of  elevated  reading  times  at 
the  five  informative  words.  As  an  indication  of  this,  we  expect 
longer  times  for  informative  words  than  for  the  words  adjacent  to 
them.  In  the  antecedent-first  condition,  this  criterion  was  met 
only  for  the  two  informative  words  that  ended  components;  in  the 
action-first  and  consequence-first  conditions,  the  criterion  was 
met  for  3  of  the  5  informative  words.  Thus,  there  is  at  best 
limited  support  for  this  hypothesis. 

2.3.4  Interpretation 

It  appears  that  end-o f -c omponent  words  and  informative  words 
are  responsible  for  only  part  of  the  variation  in  reading  times. 
Another  factor  that  seems  to  be  involved  is  serial  position  in 
the  instruction:  subjects  tend  to  pause  at  certain  points  when 
reading  the  instruction  regardless  of  what  word  occurs  at  that 
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position.  As  evidence  for  this,  there  are  positive  correlations 
between  the  reading  times  for  any  two  of  our  conditions  over  the 
positions  of  the  sentence,  e.g.,  it  takes  longer  to  read  the 
tenth  than  the  eleventh  word  in  the  sentence,  regardless  of  what 
exact  words  fill  those  pos i t i ons .  Thi s  pos i t i on  e f f ec t  1  ike  1 y 
reflects  a  rehearsal  process:  because  they  must  recall  the 
instruction  verbatim,  subjects  pause  at  various  points  to 
rehearse  what  they  have  read  thus  far,  and  these  rehearsal  points 
ere  c  -termined  by  how  much  is  in  memory  not  by  what  the  current 

word  i  s  . 
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3.  SCHEMAS  AND  A  COMPLEX  ASSEMBLY  TASK 

3.1  Rationale 

Experiment  1  showed  that  when  subjects  had  to  understand  an 
instruction,  reading  time  was  fastest  when  the  topics  were 
mentioned  in  the  order  they  needed  to  be  executed.  To  extend  the 
generality  of  this  finding  and  to  provide  some  further  tests  of 
our  schema  account  of  it,  we  decided  to  switch  from  an  operation 
to  an  assembly  task,  and  to  use  a  task  where  the  steps  differed 
appreciably  in  the  number  of  topics  or  slot-fillers  they 
contained . 
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The  task  we  used  involved  assebling  a  model  helicopter  from 
Fi scher-Technik  parts.  The  written  instructions  to  do  this 
contained  72  steps  of  the  form  "Slide  another  slanted  joint  onto 
the  block  so  that  the  narrow  end  faces  up".  Within  any  step,  the 
slot-fillers  might  include  an  action  (e.g.,  "slide"),  and  actor 
(e.g.,  "joint"),  and  object  (e.g.,  "block"),  a  1  ocat i on  ( e.g ., 
"knob  of  the  block"),  a  modal i tv  ( e.g .,  "so  that  narrow  end  faces 
up"),  and  an  or i entat i on  (e.g.,  "lengthwise").  To  assess  our 
order-of-ment i on  effect,  we  varied  whether  the  action,  actor  and 
object  appeared  in  the  early  or  latter  part  of  the  step;  since 
these  three  slot-fillers  are  needed  early  in  execution,  we 
expected  faster  reading  times  when  they  occurred  early  in  the 
step.  To  further  evaluate  our  schema  hypothesis,  we  varied  the 
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number  of  slot-fillers  in  a  step;  the  mce  slots  to  be 
instantiated,  the  longer  it  should  take  to  read  the  step. 

3.2  Experiment  3:  Assembling  a  Helicopter 

3.2.1  Method 

3 . 2 . 1 . 1  Overvi ew 

The  experiment  included  three  stages.  In  the  first,  we 
measured  individual  differences  in  memory.  The  second  stage  was 
devoted  to  familiarizing  the  subjects  with  the  various  components 
of  the  assembly.  The  third  stage  consisted  of  the  main  task — 
construction  of  the  helicopter.  We  discuss  these  stages  in 
sequence . 

3. 2. 1.2  Individual  differences 

Again  we  assessed  two  different  measures  of  working-memory, 
backward-digit  span  and  work ing-ro .mory  span. 

3.2. 1.3  Familiarization  with  components 

All  told,  54  parts  were  used  in  the  assembly,  which  were  of 
24  different  types.  The  names  of  these  24  types  were  quite 
transparent  (e.g.,  "small  block",  "small  plate",  "slanted 
joint").  They  had  been  taken  from  Baggett’s  (e.g.,  1985)  work 

with  the  same  helicopter  assembly;  she  had  devised  the  names  by 
first  having  subjects  generate  candidate  names,  then  using  these 
names  in  an  identification  task,  and  then  refining  the  names 
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further.  Transparent  though  the  names  were,  they  and  their 
referents  were  still  somewhat  unfamiliar  to  our  subjects,  and  so 
we  included  familiarization  training  with  the  parts  and  their 
names . 

Instances  of  the  24  different  types  and  their  names  were 
arrayed  on  a  large  piece  of  cardboard.  The  array  consisted  of  2 
columns,  each  containing  12  slots,  where  a  slot  contained  an 
actual  part  and  the  name  of  that  part.  Subjects  studied  the 
array  at  their  own  rate  (usually  on  the  order  of  3-5  minutes), 
starting  with  the  left  column  and  going  from  top-to-bottom  within 
a  column.  Typically,  they  picked  up  each  piece  and  examined  it. 
To  insure  that  they  had  learned  the  parts,  the  array  was  removed 
from  view  and  subjects  were  given  a  quiz.  The  quiz  consisted  of 
thirteen  questions  (only  half  the  parts  were  queried),  where  the 
typical  question  first  asked  the  subject  to  find  a  part  with  a 
particular  property  or  name,  and  then  to  state  how  this  part 
differed  from  similar  ones.  A  few  of  the  questions  also  required 
a  minor  assembly  operation  (e.g.,  putting  together  a  tire  and 
wheel).  If  subjects  made  a  mistake,  they  were  informed  of  their 
error  and  told  to  try  again.  Subjects  typically  made  few  errors. 

3.2. 1.4  Assembly  task 

The  heart  of  the  experiment  required  assembling  the 
helicopter  from  written  instructions.  A  diagram  of  the 
helicopter,  which  gives  its  major  subassemblies,  is  presented  in 
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Figure  4.  There  were  72  steps,  which  had  the  subjects  build  the 
copter  from  the  tail  section  to  the  main  propeller.  For  half  the 
subjects,  the  steps  were  such  that  the  action,  actor,  and  object 
were  mentioned  first,  while  information  about  location,  modality, 
and  orientation  were  mentioned  in  the  second  part  of  the  step; 
this  is  referred  to  as  the  "basic"  order.  For  the  other  half  of 
the  subjects,  the  order  of  mention  was  reversed  so  that 
information  about  location,  modality,  and  orientation  came  first; 
this  is  referred  to  as  the  "modified"  order.  For  both  orders, 
some  of  the  steps  mentioned  3  slot-fillers,  others  mentioned  4, 
and  the  rest  mentioned  5.  The  steps  therefore  represented  the 
orthogonal  combination  of  order  of  slot-fillers  (basic  vs. 
modified)  and  number  of  slot-fillers  (3  vs.  4  vs.  5).  Examples 
of  all  six  kinds  of  steps  are  shown  in  Figure  5.  However,  the 
set  did  not  contain  an  equal  number  of  each  type. 

In  addition  to  these  steps,  we  also  included  in  the 
instructions  statements  that  introduced  each  subassembly  of  the 
helicopter  (e.g.,  "First  you  will  assemble  the  tail  section  of 
the  helicopter.  This  section  includes  the  rear  propeller".),  as 
well  as  statements  that  signalled  completion  of  a  subassembly 
(e.g.,  "Make  sure  the  rear  propeller  turns.  If  it  does 
—  congratulations!!  -  you  have  successfully  completed  the  tail 
section.")  These  statements  provide  knowledge  about  the  assembly 
other  than  that  contained  in  the  steps  per  se .  but  there  was  no 
variation  in  this  other  knowledge.  Similarly,  in  addition  to  the 
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Illustrations  of  Materials  in  Helicopter  Assembly  Task 


#  Topics 
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#  Topics 

3 

4 


5 


Basic  Instructions 

Tighten  the  handle  by  turning  it  clockwise, 


Slide  another  slanted  joint  onto  the  block 

SO  THAT  THE  NARROW  END  FACES  UPWARD, 


Modified  Instructions 

By  turning  clockwise,  tighten  the  handle. 


With  the  narrow  end  facing  upward,  slide 

ANOTHER  SLANTED  JOINT  ONTO  THE  BLOCKS 
KNOB, 


With  the  knob  facing  right  and  the  slot  on 

THE  END  OF  THE  BLOCK  VERTICALLY,  LAY  A 
LARGE  BLOCK  LENGTHWISE  ON  THE  TABLE, 


Uy  a  U\RGE  BLOCK  LENGTHWISE  ON  THE  TABLE  SO 
THAT  THE  KNOB  FACES  TO  THE  RIGHT  AND  THE 
SLOT  ON  THE  END  OF  THE  BLOCK  IS  VERTICAL, 
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instructions  we  also  provided  subjects  a  diagram  of  two  views  of 
the  completed  helicopter  (see  Figure  6),  which  remains  in  view 
throughout  the  task.  This  diagram  also  provides  assembly 
knowledge  additional  to  the  instructions,  but  again  we  did  not 
experimentally  manipulate  any  aspect  of  this  additional 
knowl edge . 

The  details  of  the  procedure  are  as  follows.  Instructions 
appeared  one  step  at  a  time  on  a  microprocessor  display.  The 
name  of  the  current  subassembly  appeared  in  the  lower  right  of 
the  screen;  we  did  this  because  often  we  had  to  use  the  name  of 
the  subassembly  in  the  step,  and  also  because  Baggett  suggests 
that  this  makes  things  easier  for  the  subjects.  Subjects  read 
each  instruction,  pressed  a  space  bar  when  they  thought  they 
understood  it,  and  tried  to  execute  the  instruction  after  it 
disappeared  from  view  (the  assembly  kit  was  adjacent  to  the 
display).  Both  reading  time  and  execution  time  were  recorded  for 
each  step. 

3.2.2  Results 

The  results  of  interest  are  all  from  the  assembly  task. 

3. 2. 2.1  Major  results 

Raw  reading  times  varied  from  about  5  to  25  seconds.  Since 
the  number  of  syllables  in  a  step  covaried  with  the  number  of 
topics  or  slot-fillers,  we  converted  each  raw  reading-time  to  a 
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reading  time  per-sy 1 1 ab 1 e .  These  normalized  scores  are  the 
referents  of  "reading  times"  in  what  follows. 

Figure  7  presents  reading  times,  as  a  function  of  number  of 
slot-fillers,  separately  for  basic  and  modified  instructions. 
Steps  are  read  faster  with  basic  than  modified  instructions, 
which  extends  our  earlier  finding  that  reading  is  facilitated 
when  the  order  of  mention  mirrors  the  order  of  execution. 
Moreover,  for  both  sets  of  instructions,  reading  times  increase 
monotoni cal ly  with  the  number  of  slot-fillers  in  the  step.  This 
result  provides  an  added  reason  for  construing  the  process  of 
understanding  a  step  as  instantiating  the  slots  of  a  step  schema. 
Furthermore,  there  is  an  interaction  between  order  and  number  of 
s 1 ot -f i 1 1 ers  ,  such  that  the  number  has  a  more  deleterious  effect 
with  a  non-opt imal  ordering  than  an  optimal  ordering  of 
slot-fillers. 

Figure  8  depicts  the  same  results  as  above,  but  presented 
separately  for  subjects  with  different  working-memory  spans  (as 
estimated  by  the  Daneman  and  Carpenter,  1980,  technique).  Note 
that  the  time  scale  is  expanded  for  the  subjects  with  the  largest 
spans.  In  general,  the  higher  their  span,  the  faster  the 
subjects  read  the  steps.  More  interestingly,  high-span  subjects 
showed  less  effect  of  the  number  of  slot-fillers  than  did 
subjects  with  lower  spans. 

Figures  9  and  10  present  the  comparable  analyses  of 
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execution  times.  Since  the  number  of  syllables  in  a  step  was 
related  to  the  number  of  actions  (including  checking  actions) 
needed  to  execute  the  step,  we  again  normalized  each  score  by  the 
number  of  syllables  in  the  step.  With  regard  to  the  data  in 
Figure  9,  there  was  no  overall  effect  of  either  number  of 
slot-fillers  or  basic  vs.  modified  instruction;  but  there  was  an 
interaction  between  the  two  variables  such  that  the  basic 
instructions  took  longer  than  the  modified  ones  only  when  the 
step  contained  four  slot-fillers.  This  anomalous  result  occurred 
at  all  sizes  of  working-memory  span  (see  Figure  10). 

3.2.2. 2  Other  results 

Three  other  findings  are  worth  briefly  noting.  First,  when 
backward-digit  span  is  used  as  the  measure  of  working  memory,  the 
regularities  between  working-memory  and  RT  disappear.  That  is, 
in  contrast  to  the  Daneman  and  Carpenter  measure,  there  is  no 
relation  between  backward-digit  span  and  the  comprehension  of 
i ns  true  t i ons  . 

Second,  there  were  relatively  few  errors  of  execution; 
indeed,  for  many  of  the  steps,  no  subject  ever  made  an  error. 
The  errors  that  did  occur  seemed  tied  to  the  specifics  of  the 
step,  and  there  was  no  difference  in  the  distribution  of  errors 
between  the  basic  and  modified  instructions  —  see  Figure  11. 

Third  and  finally,  an  analysis  of  subjects’  inspections  of 
the  available  diagram  showed  that  there  was  no  relation  between 
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the  number  of  times  a  subject  checked  the  diagram  and  how  quickly 
that  subject  read  or  executed  the  steps.  However,  all  subjects 
did  check  the  diagrams  more  on  steps  that  were  error  prone;  this 
is  indicated  in  Figure  12,  which  shows  similar  patterns  to  those 
observed  in  our  analysis  of  errors.  Figure  12  also  indicates 
that  there  was  no  difference  in  the  distribution  of  diagram 
checks  between  the  basic  and  modified  instructions. 

3.2.3  Interpretation  of  Major  Results 


With  regard  to  the  reading-time  results,  we  suspect  that  the 
effects  of  order  of  topics  or  slot-fillers,  number  of 
slot-fillers,  and  working-memory  span  all  have  the  same 
theoretical  locus,  namely  instantiating  a  step  schema.  Assume 
that  understanding  a  step  involves  assigning  each  topic  in  the 
step  to  a  slot  in  a  step-schema,  where  these  slots  include  actor . 
action,  object,  orientation.  location,  and  modal i tv .  Assume 
further  that  this  instantiation  process  takes  place  in  working 
memory,  which  is  very  limited  with  respect  to  how  much  material 
can  be  kept  active  at  any  point  in  time  (e.g.,  Anderson,  1976). 
The  more  schema  slots  to  be  filled  in  a  step,  the  less  activation 
there  will  be  for  any  slot  filler,  and  the  longer  it  will  take  to 
understand  the  step.  Hence  reading  time  should  increase  with  the 


number  of  slot-fillers 


Moreover,  the  smaller  one’s 


work i ng-memory  span,  presumably  the  less  activation  one  has,  and 
the  more  deleterious  the  effect  of  extra  slot-fillers.  Hence  the 
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interaction  we  observed  between  working-memory  span  and  number  of 
slot-fillers.  Finally,  if  fillers  must  be  assigned  to  slots  in  a 
particular  order  —  namely,  actor .  action,  and  object  first, 
because  they  are  the  top-level  slots  —  then  when  the  fillers  are 
mentioned  in  some  other  order,  some  of  them  may  have  to  be 
temporarily  stored  rather  than  instantiated,  and  this  should  thin 


the  activation 

still  further . 

This  should 

lead 

to 

order-of-f i 1 1 ers 

interacting  with  both 

and  ( 1 )  the 

number 

of 

fillers  and  (2)  working-memory  span;  we  observed  the  first  of 
these  interactions,  but  not  the  second. 

Finally,  some  comment  should  be  made  about  the  fact  that 
variables  that  affected  reading  times  had  little  or  no  influence 
on  execution  times.  This  pattern  has  emerged  before  in  our  work; 
e.g.,  in  Experiment  1,  which  dealt  with  operating  a  simple 
device,  the  order  of  slot-fillers  affected  reading  time  but  not 
execution  time.  We  take  this  pattern  to  mean  that  the  variables 
of  interest  primarily  affect  the  understanding  process,  which  is 
what  the  reading-time  measure  reflects. 
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4.  MENTAL  MODELS  AND  AN  OPERATIONS  TASK 

4.1  Rati onal e 

We  now  shift  our  focus  to  knowledge  other  than  that 
contained  in  the  instructions.  In  our  previous  studies  with  this 
focus  (Smith  &  Goodman,  1984),  we  used  an  assembly  task,  and  the 
additional  knowledge  was  such  that  it  would  presumably  be 
represented  as  a  schema  (the  information  rarely  referred  to 
change  and  was  easily  represented  propositionality).  In  the 
current  study,  in  contrast,  we  used  a  device-operation  task,  and 
the  additional  knowledge  was  such  that  it  presumably  was 
represented  as  a  mental  mode  1  of  the  device  (the  information 
encompassed  dynamic  and  changing  properties  of  a  system). 

The  task  required  using  multi-step  procedures  to  operate  the 
display  illustrated  in  Figure  13.  This  task  is  an  elaboration  of 
one  developed  by  Kieras  and  Bovair  (1984),  and  our  work  rests  on 
theirs  in  a  number  of  respects.  Kieras  and  Bovair  have  shown 
that  providing  subjects  a  mental  model  of  the  device  shortened 
the  number  of  trials  that  subjects  needed  to  learn  procedures  for 
operating  the  device.  Our  concerns  were  somewhat  different. 
First,  we  wanted  to  determine  whether  having  a  model  of  the 
device  facilitates  the  comprehension  of  each  instructional  step 
in  a  multi-step  procedure,  as  reflected  in  the  time  to  read  each 
step.  Thus,  we  were  interested  in  the  comprehension  of  each  step 
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rather  than  the  learning  of  the  entire  procedure.  Second,  we 
wanted  to  extend  the  generality  of  our  previous  findings  and  show 
that  the  order  in  which  the  topics  were  mentioned  affected 
performance  in  the  current  task;  hence  there  was  some  concern 
with  the  instructions  per  s e  as  well  as  with  other  knowledge  in 
the  current  study.  Third  and  finally,  we  were  interested  in  how 
knowledge  in  the  form  of  a  model  affected  reasoning  about  the 
device. 

4.2  Experiment  4:  Operating  a  Device  via  Multi-Step  Procedures 

4.2.1  Method 

4 . 2 . 1 . 1  Overview 

The  experiment  included  four  stages.  The  first  three  were 
similar  to  those  of  the  previous  study.  Thus,  stage  1  measured 
individual  differences,  stage  2  familiarized  subjects  with  the 
components  or  their  task,  as  well  as  introduced  the  model,  and 
stage  3  consisted  of  the  main  task  of  operating  the  device.  In 
stage  4,  subjects  were  given  the  equivalent  of  a  couple  of  steps 
and  asked  to  generate  the  rest  of  the  procedure.  This  stage  was 
included  so  as  to  provide  another  opportunity  to  observe  the 
effects  of  mental  model  on  the  use  of  instructions. 


4. 2. 1.2  Individual  differences 
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4.2. 1.3  Familiarization  with  components  and  presentation  of  the 
mode  1 

With  the  display  in  front  of  them,  subjects  were  told  about 
the  various  components  of  the  display.  Exactly  what  they  were 
told  depended  on  whether  subjects  were  given  a  model  of  the 
device  (hereafter,  the  "model"  group),  or  not  (the  "control" 
group).  Subjects  in  the  control  group  were  told  the  following: 

"Before  explaining  the  components  of  the  system,  I  want  to 
tell  you  that  your  goal  when  operating  the  device  is  to  get  the 
PFI  light  to  llash  four  times,  which  will  enable  you  to  push  the 
PF  button.  Now  we  can  go  on  to  the  components  of  the  device. 

The  first  component  is  the  SP  switch.  It  has  two  positions. 
When  the  SP  switch  is  flipped  up,  the  SPI  is  illuminated.  The 
device  will  not  work  unless  the  SP  is  flipped  up.  Thus,  the 
neutral  position  for  the  SP  switch  is  the  down  position.  The 
second  component  is  the  ELS  control.  It  is  a  sliding  selector  on 
a  logarithmic  scale  that  ranges  from  0  to  100.  The  third 
component  is  the  AS  control  It  has  three  positions  -  MA ,  SA ,  and 
N.  When  the  AS  is  turned  to  MA ,  the  MAI  is  illuminated.  When  the 
AS  is  turned  to  SA ,  the  SAI  is  illuminated.  When  the  AS  is 
turned  to  neither  control  (N),  then  neither  the  MAI  nor  the  SAi 
will  be  illuminated.  The  AS  is  important  because  the  PFI  will 
not  flash  unless  the  MA  or  SA  setting  is  selected.  The  neutral 
position  for  the  AS  is  the  N  position.  The  setting  of  the  AS 
selector  is  related  to  that  of  the  ELS  control,  in  that  settings 
of  the  ELS  above  50  require  selection  of  the  MA  position  in  order 
for  the  device  to  work. 

The  fourth  component  is  the  TC  dial.  It  is  a  circular 
control  and  has  a  range  from  0  to  360.  A  TC  setting  of  more  than 
180  is  compatible  only  with  the  AS  control  being  at  MA ;  if  the  TC 
setting  is  180  or  less,  then  the  AS  control  can  be  at  either  MA 
or  SA .  The  TC  is  important  because  the  PFI  will  not  flash  until 
this  dial  is  set  correctly.  The  fifth  and  sixth  components  are 
the  PFI  and  the  PF  button,  respectively.  The  PFI  will  flash  four 
times.  This  means  that  you  can  push  the  PF  button  and  reach  your 
goal.  After  you  have  done  so,  the  controls  should  be  placed  in 
the  neutral  position,  that  is,  the  SP  switch  should  be  flipped 
down  and  the  AS  should  be  set  to  N." 

In  contrast,  the  model  group's  familiarization  with  the 
components  included  a  description  of  a  device  model 
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"The  device  you  will  operate  is  the  phaser  control  panel  of 
the  starship  Enterprise.  The  phaser  bank  firing  system  can  be 
divided  into  three  subsystems:  Power  Booster,  Power  Accumulator 
and  Phaser  Firing.  The  functions  of  the  system  components  will 
be  explained  in  terms  of  these  three  sub-systems. 

The  Power  Booster  subsystem  consists  of  three  components, 
the  Shipboard  Power  Switch  (SP),  the  Shipboard  Power  Indicator 
(SP1),  and  the  Energy  Booster  (EB) .  When  the  SP  switch  is 
flipped  up,  the  SPI  is  illuminated.  Energy  flows  from  the 
shipboard  engines  to  the  Energy  Booster  (EB) .  The  EB  boosts  the 
power  from  the  ship's  engines  to  a  higher  level.  Power  will  not 
be  accumulated  in  the  accumulators  if  the  EB  malfunctions. 

The  Power  Accumulator  subsystem  consists  of  the  Energy  Level 
Selector  (ELS),  the  Accumulator  Selector  (AS)  and  the  latter's 
related  components,  the  Main  Accumulator  (MA) ,  the  Secondary 
Accumulator  (SA),  the  Main  Accumulator  Indicator  (MAI),  and  the 
Secondary  Accumulator  Indicator  (SAI).  The  Energy  Level  Selector 
(ELS),  a  sliding  selector  on  a  logarithmic  scale  that  ranges  from 
0  to  100,  determines  the  amount  of  power  that  will  be  stored  in 
one  of  the  accumulators.  The  Accumulator  Selector  (AS)  allows 
one  to  choose  which  accumulator  will  receive  and  store  the  power. 
The  Main  Accumulator  (MA)  collects  10  Universal  Power  Units 
(UPU's)  per  second  while  the  Secondary  Accumulator  (SA)  collects 
5  UPU's  per  second.  When  the  MA  is  selected,  the  Main 
Accumulator  Indicator  (MAI)  is  illuminated.  When  the  SA  is 
selected,  the  Secondary  Accumulator  Indicator  (SAI)  will  be 
illuminated.  If  Neither  (N)  accumulator  is  selected  (neutral 
position),  both  indicators  will  be  off.  Energy  will  not  be 
stored  in  the  accumulators  when  the  AS  is  in  the  N  position.  The 
phasers  cannot  be  fired  unless  an  accumulator  (MA  or  SA)  is 
selected.  The  setting  of  the  Energy  Level  Selector  and  the 
accumulator  selected  depend  on  the  distance  between  the 
Enterprise  and  the  target.  For  distant  targets,  the  ELS  must  be 
set  above  50;  since  the  amount  of  power  that  must  be  stored  is 
large,  the  Accumulator  Selector  must  be  set  at  the  MA  position. 

The  third  subsystem,  Phaser  Firing,  consists  of  the  Target 
Coordinator  (TC),  the  Phaser  Firing  Button  (PF),  the  Phaser 
Firing  Indicator  (PFI),  and  the  Phaser  BAnk  (PB) .  The  Target 
Coordinator,  a  circular  control,  specifies  the  angular  direction 
of  the  target  from  the  Enterprise  in  degrees,  ranging  from  0  to 
360.  For  targets  between  0  to  180  degrees,  only  limited  power  is 
needed  so  either  the  Secondary  or  Main  Accumulator  can  be  used; 
for  targets  at  an  angular  direction  of  greater  than  160,  more 
power  is  needed,  so  the  Accumulator  Selector  must  be  set  at  the 
MA  position.  When  enough  power  has  been  stored  in  an  appropriate 
accumulator,  the  PFI  will  flash  four  times  if  the  TC  is  set 
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correctly.  After  the  PFI  light  flashes  four  times,  the  PF  button 
should  be  pushed.  This  releases  power  from  the  Accumulator  to 
the  Phaser  Bank,  firing  the  phasers  in  the  direction  specified  by 
the  Target  Coordinator.  After  the  phasers  have  been  fired,  the 
controls  should  be  placed  in  the  neutral  position,  that  is  the  SP 
switch  should  be  flipped  down  and  the  AS  should  be  set  to  N. 

After  hearing  one  of  the  above  explanations,  the  device  was 
removed  from  view,  and  subjects  in  both  the  model  and  control 
groups  were  given  a  ten-question  quiz  about  the  components  and 
their  settings.  The  question  asked  either  for  the  name  of  a 
component  that  had  a  particular  property,  or  for  a  property  of  a 
named  component,  or  for  both  (eg- ,  "What’s  the  name  of  the 
switch  with  two  positions?  What  are  the  two  positions?").  If 
subjects  made  any  errors  on  the  quiz,  the  entire  explanation  was 
repeated  and  they  were  retested  on  the  items  that  they  had 
missed. 

4. 2. 1.4  Instructions  task 

The  main  part  of  the  experiment  involved  following 
multi-step  procedures  for  operating  the  device.  Subjects  worked 
with  4  basic  4-step  procedures,  and  they  saw  each  procedure  3 
times.  Two  of  the  basic  procedures  are  given  in  Figure  14.  Note 
that  in  each  procedure  three  of  the  steps  are  "simple",  in  that 
they  specify  a  single  action,  while  one  step  is  "complex",  in 
that  it  specifies  three  actions;  this  distribution  of  simple  and 
complex  steps  held  for  all  procedures.  The  reason  for  including 
the  complex  steps  was  to  allow  a  variation  in  the  ordering  of 
topics  or  slots;  e.g.,  in  Figure  14,  the  topics  are  mentioned  in 
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L  Flip  the  SP  up 

2,  Set  the  ELS  to  100 

3,  Adjust  the  AS  to  MIA 

A,  In  Order  to  Get  the  PFI  to  Fush  Four  Times  so  that  You  Can  Push  the 
PF  button  —  Adjust  the  TC  to  260 

’-#r 

1,  Flip  the  SP  on  and  Adjust  the  ELS  to  1  and  then  Turn  the 
AS  ToflA 

2,  Set  the  TC  to  250 

3,  Wait  Unitl  the  PFI  Flashes  Four  Times 

4,  Push  the  PF  Button 
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the  order  of  execution  in  the  first  procedure  but  not  in  the 
second . 

The  four  basic  procedures  varied  in  several  respects:  (1) 
the  complex  step  was  either  first  or  last;  (2)  the  order  of  the 
topics  in  the  complex  step  was  either  optimal  or  nonoptimal 
(i.e.,  order  of  mention  either  mirrors  execution  or  not);  and  (3) 
the  steps  included  either  (a)  the  SA  setting  on  the  Accumulator 
Selector  and  compatible  settings  on  the  ELS  (below  50)  and  the  TC 
(below  180),  or  (b)  the  MA  setting  on  the  Accumulator  Selector 
and  arbitrary  settings  or  the  ELS  and  TC  (this  captures  the 
constraints  between  the  Accumulator  Selector,  ELS,  and  TC)  . 

Each  procedure  was  presented  one  step  at  a  time  on  the 
display  of  microprocessor.  Subjects  read  each  step,  pressed  a 
space  bar  when  they  thought  they  understood  it,  and  tried  to 
execute  the  instruction  after  it  disappeared  from  view  (the 
device  was  adjacent  to  the  display).  Both  reading  time  and 
execution  time  were  recorded  for  each  step. 

4. 2. 1.5  Generation  task 

The  purpose  of  this  task  was  to  explore  how  the  model 
affected  subjects’  reasoning  about  the  procedures.  A  total  of  12 
partial  procedures  were  presented,  each  containing  information 
about  1,2,  or  3  of  the  6  acts  needed  to  specify  a  complete 
procedure.  Samples  are  presented  in  Figure  15.  Subjects  had  to 
determine  how  to  complete  the  procedure,  and  they  were  asked  to 
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Get  the  PFI  to  Flash  in  a  Short  Amount  of  Time  While  the  TC  is 
set  at  35 

The  ELS  Should  be  Adjusted  to  a  High  Value  and  the  TC  Turned 
to  310 

Choose  Compatible  Settings  For  All  of  the  Controls  if  the  TC 
is  Set  at  220 
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think  aloud  while  they  did  this.  Then  they  tried  to  execute 
their  planned  procedure.  The  partial  procedures  always  specified 
the  TC  setting,  but  never  gave  all  three  of  the  settings  that  had 
constraints  between  them. 

4.2.2  Results 

Three  different  stages  yielded  results  of  interest, 
including  component  familiarization,  the  instructions  task,  and 
the  generation  task.  We  consider  them  in  turn. 

4.2.2. 1  Component  f ami  1 ar  i  zat  i  on 

Recall  that  this  stage  involved  a  ten-question  quiz  on  the 
names  and  properties  of  the  components.  Figure  16  presents  the 
average  number  of  errors  subjects  made  as  a  function  of  their 
working-memory  span  and  whether  they  were  in  the  model  or  control 
group.  Among  high-span  subjects,  there  is  no  difference  between 
the  number  of  errors  in  the  model  and  control  groups;  among 
low-span  subjects,  there  is  a  very  substantial  difference,  with 
model  subjects  making  many  fewer  errors.  This  boost  for  the 
low-span  subjects  is  consistent  with  meaningfulness  increasing 
memory.  Thus,  the  names  of  components  are  essentially  nonsense 
syllables  to  the  control  subjects  but  acronyms  of  meaningful 
units  to  the  model  subjects,  and  the  addition  of  "meaning"  to  a 
nonsense  syllable  boosts  its  memor a  1 i b i 1 i ty  (e.g.,  Crowder, 
1976),  a  boost  that  is  sorely  needed  by  subjects  with  low  spans. 
While  this  result  does  not  tell  us  much  about  the  effects  of 
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mental  models  on  comprehension,  its  impact  ought  not  to  be 
minimized  —  learning  the  names  and  properties  of  task-relevant 
components  can  be  a  major  source  of  difficulty,  and  if  a  mental 
model  reduces  this  difficulty  then  this  is  an  important  aspect  of 
a  model . 

4. 2. 2. 2  Instructions  task 

The  major  result  to  emerge  from  this  experiment  is  that 
having  a  model  facilitated  the  reading  of  steps. 

Figure  17  presents  r ead i ng-t ime-pe r-sy 1  1  ab  1 e  ,  in  seconds, 
for  simple  and  complex  steps,  separately  for  the  model  and 
control  groups.  After  normalization  by  number  of  syllables, 
complex  steps  are  read  faster  than  simple  ones,  suggesting  that 
subjects  speed  up  as  they  go  through  the  step,  perhaps  because 
they  can  infer  some  of  the  later  information.  There  is  also  a 
beneficial  effect  of  having  a  model,  which  is  worth  about  30  msec 
per  syllable.  Figure  18  presents  the  comparable  data  for 
execution  time  pe r-sy 1 1 ab 1 e .  There  is  less  effect  of  complexity 
on  the  normalized  times,  and  no  effect  at  all  of  having  a  model. 

Figure  19  plots  r ead i ng-t ime-pe r-sy 1 1 ab 1 e  as  a  function  of 
working-memory  span,  separately  for  the  model  and  control  groups. 
There  is  no  effect  of  span,  which  is  not  surprising  given  that 
most  of  the  steps  were  short  and  simple.  As  before,  there  is 
about  a  30-40  msec  benefit  per  syllable  when  subjects  have  a 
model.  Figure  20  presents  the  comparable  data  for  execution 
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times  There  is  no  effect  of  span,  nor  is  there  any  beneficial 
effect  of  the  mode  1  . 

Figures  21  and  22  consider  only  complex  steps,  and  focus  on 
the  effects  due  to  differential  orderings  of  the  topics  or  slots 
within  a  step.  Figure  21  plots  reading-time-per-syl lable  as  a 
function  of  optimal  vs.  non-optimal  orderings,  separately  for 
the  model  and  control  groups.  Unlike  the  results  in  our  previous 
experiments,  there  is  no  significant  effect  of  order  on  reading 
time  in  the  present  case;  having  a  model  again  facilitates 
reading,  though,  with  the  effect  this  time  being  about  50  msec 
per  syllable.  The  comparable  results  for  execution  times  are  in 
Figure  22.  Now,  there  is  a  substantia]  effect  of  the  order  of 
topics,  with  the  optimal  order  resulting  in  a  marked  reduction  in 
execution  time,  especially  for  the  control  group;  while  there  is 
no  overall  effect  of  having  a  model,  there  is  an  interaction 
between  presence  of  a  model  and  type  of  instruction. 

In  addition  to  its  effects  on  the  speed  of  execution,  having 
a  model  also  improved  the  accuracy  of  execution.  The  average 
number  of  errors  was  4.38  in  the  model  group,  compered  to  5.85 
errors  in  the  control  group. 

To  summarize  the  major  results:  there  is  a  small  but 
consistent  effect  of  having  a  model  on  reading  times,  and  a 
substantial  effect  of  ordering  on  execution  times.  These  results 
raise  two  questions  of  interpretation:  Why  does  ordering  affect 
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execution  but  not  reading?,  and  How  does  having  a  model  lead  to 
improvements  in  reading? 

With  regard  to  the  question  about  ordering,  in  our  previous 
experiments  using  an  operation  task  (Experiment  1)  or  an  assembly 
task  (Experiment  3),  we  found  that  the  ordering  of  the  topics  or 
slot-fillers  affected  reading  time  but  had  no  effect  on  execution 
time.  In  interpreting  these  findings,  we  assumed  that:  (l)  the 
ordering  effects  reflects  understanding  processes  (e.g., 
instantiating  a  step  schema),  and  (2)  these  processes  are 
completed  before  execution  begins,  i.e.,  subjects  do  not  attempt 
to  execute  an  action  until  their  plan  for  that  action  is 
completed.  We  suspect  that  the  present  results  —  that  ordering 
affects  execution  times  but  not  reading  times  —  reveal  a  failure 
of  our  second  assumption.  In  the  present  study  as  in  earlier 
ones,  an  optimal  ordering  of  the  topics  or  slot-fillers  may  have 
facilitated  comprehension  of  the  step,  but  this  time  subjects 
started  to  execute  the  actions  before  they  fully  understood  them, 
and  hence  some  comprehension  effects  spilled  over  into  the 
execut i on  phase  . 

Consider  now  the  question  of  how  a  model  leads  to 
improvements  in  reading.  One  possibility  is  that  model  subjects 
were  more  familiar  with  the  components  than  control  subjects. 
However,  we  suspect  this  factor  was  of  limited  influence  because 
even  control  subjects  continued  component  f ami  1 i ar  i  zat  i  on  until 
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they  knew  all  the  items.  A  more  likely  possibility  is  suggested 
by  Kieras  and  Bovair  (1984).  They  propose  that  a  model 
facilitates  the  learning  of  procedures  by  encouraging  inferences; 
more  inferences  may  have  played  a  role  in  our  results  too. 
Similarly,  the  model  may  have  allowed  subjects  to  see  the 
relations  between  the  steps  in  a  procedure,  and  even  though 
subjects  did  not  need  these  relations  to  perform  their  task  they 
may  have  tried  to  understand  them.  We  found  some  evidence  for 
this  kind  of  mechanism  in  our  earlier  studies  showing  a 
beneficial  effect  of  explanatory  material  (Smith  &  Goodman, 
1984)  . 

We  can  flesh  out  the  last  two  proposals  with  an  example. 
Suppose  the  second  step  of  a  procedure  is  "Set  the  ELS  to  100". 
Subjects  in  the  model  group  might  refer  this  to  the  model,  and 
realize  that  it  suggests  a  distant  target  which  necessitates  a 
lot  of  power  be  stored.  If  the  next  step  is  "Adjust  the  AS  to 
MA,"  our  model  subjects  have  already  inferred  it;  if  a  subsequent 
step  is,  "Adjust  the  TC  to  260,"  while  model  subjects  may  not 
have  inferred  it  they  at  least  see  the  relation  between  it  and 
previous  steps.  Consider  now  subjects  in  the  control  group. 
When  presented,  "Set  the  ELS  to  100,"  they  too  have  information 
which  permits  them  to  infer  that  the  AS  will  be  set  to  MA ,  but 
this  information  is  abstract  and  correlational,  and  hence  less 
likely  to  be  used.  In  short,  mode.1  subjects  may  draw  more 
inferences  and  construct  more  relations  between  steps  because 
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they  can  use  something  like  causal  reasoning  with  concrete 
material . 

4. 2. 2. 3  Generation  task 

The  primary  measure  of  performance  in  this  stage  was  the 
average  number  of  generated  procedures  that  contained  an  error: 
this  number  was  marginally  greater  for  the  control  than  the  model 
group  —  2.23  and  1.62,  respectively  —  suggesting  that  a  model 
enhances  one’s  ability  to  reason  about  possible  procedures. 
Though  the  performance  difference  is  marginal,  it  seems  useful  to 
ask  how  it  might  have  arisen.  This  time  the  verbal  protocols 
provide  a  clue.  In  reasoning  about  the  procedures  they  had  to 
generate,  subjects  in  both  the  model  and  control  groups  typically 
mention  the  constraints  between  the  settings  on  the  Accumulator 
Selector,  Energy  Level  Selector,  and  Target  Coordinator;  but  most 
subjects  in  the  model  group,  11  of  13,  expand  on  these 
constraints  (they  do  this  by  bringing  up  considerations  about 
energy  flow),  while  few  subjects  in  the  control  group,  4  of  13, 
engaged  in  such  expansion.  Perhaps  the  greater  embellishment  in 
the  model  group  highlighted  the  importance  of  the  constraints, 
which  led  to  a  greater  use  of  them  in  the  generation  of 
procedures,  and  to  fewer  errors.  This  is  roughly  the  same 
account  we  offered  of  why  model  subjects  might  make  more 
inferences  than  control  subjects. 
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4.2.3  A  Final  Comment 


While  our  results  are  preliminary,  they  suggest  that  a  model 
produces  widespread  effects.  Having  a  model  facilitated  memory 
of  the  components,  compr < hens i on  of  instructional  steps, 
execution  (accuracy)  of  the  steps,  and,  to  some  extent,  reasoning 
about  new  procedures.  While  the  effects  were  small,  the  model 
and  task  were  quite  simple,  which  no  doubt  limited  the  influence 
the  model  could  have.  More  complex  models,  particularly  causal 
ones,  might  produce  far  more  striking  effects. 


Lastly,  this  study  shows  that  the  two  foci  of  our  research 
may  be  merged.  We  obtained  effects  of  both  the  model  and  the 
ordering  of  topics  v/ithin  a  step.  Moreover,  the  ordering  effect 
was  less  in  the  model  group  than  in  the  control,  which  suggests 
that  the  more  one  relies  on  knowledge  other  than  the  instructions 
the  less  the  efficacy  of  the  instructions  matters. 
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5 .  SUMMARY 

The  major  findings  of  our  research  include: 

(1)  For  both  operations  and  assembly  tasks,  less  time  is 
needed  to  understand  an  instructional  step  if  the  topics  are 
mentioned  in  the  order  in  which  they  will  be  executed. 

(2)  The  importance  of  or de r-o f-ment i on  is  greater  when  the 
reader's  working-memory  is  taxed;  hence,  order-of-ment i on  matters 
more  for  reader’s  with  smaller  working-memory  spans. 

(3)  To  measure  working-memory  span  in  a  way  that  is 
predictive  of  success  with  written  instructions,  one  should  not 
use  standard  memory-span  tests;  rather,  one  should  use  a  test 
that  involves  processing  as  well  as  storage,  like  the 
Daneman-Carpenter  (1980)  measure. 

(4)  Although  operations  and  assembly  tasks  may  seem  very 
different  on  the  face  of  it,  the  two  tasks  are  affected  in  the 
same  way  by  the  same  variables  (e.g.  ,  order  of  mention  within  an 
instructional  step,  role  of  explanatory  material);  this  suggests 
that  the  same  general  theory  of  instructions  may  encompass  both 
tasks  . 

(5)  The  notion  of  a  schema  can  be  successfully  applied  at 
the  level  of  a  single  instructional  step;  in  particular, 
understanding  a  step  can  be  modelled  as  instantiating  a  step 
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schema ,  thereby  accounting  for  why  reading  time  increases  with 
the  number  of  topics  mentioned  in  a  step. 

(6)  Having  a  mental  model  of  a  device  has  a  substantial 
benefit  on  memory  of  the  device’s  components,  and  smaller 
benefits  on  (a)  comprehension  of  instructions  about  using  the 
device  and  (b)  reasoning  about  new  ways  to  use  the  device. 

(7)  The  more  knowledge  other  than  the  instructions  that  one 
has  about  a  device,  the  less  affected  one  will  be  by  the  goodness 
of  the  instructions. 
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